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Abstract 

This paper suggests an approach to automatic soft- 
ware design in the domain of graphical user interfaces. 
There are still some drawbacks in existing UIMSs 
; which basicly offer only quantitative layout specifica- 
tions via direct manipulation. Our approach suggests 
; a convenient way to get a default graphical user inter- 
face which may be customized and redesigned easily 
in further prototyping cycles. 

1 Introduction 

The automation of software design becomes more 
powerful if the target systems generated are limited 
to a certain domain. The domain addressed in this 
paper is the class of graphical, highly interactive sys- 
tems for accessing data of specified data structures 
by end users. The focus of this paper is further re- 
stricted. It concentrates on the automation of the 
design of a graphical user interface (GUI) for these 
systems. 

Building GUIs with GUI toolkits or user interface 
management systems (UIMSs) is still a laborious, 
time-consuming task even if it is supported by di- 
rect manipulation facilities [6]. The basic problems 
we identified are the following: 

• The GUI designer has to decide which graphi- 
cal element is appropriate for a desired interac- 
tion, Le. given a data structure and data type 
descriptions of the elements to be accessed and 
a set of GUI dements the designer has to per- 
form a mapping between the data structure and 
the GUI dements. 

• With direct manipulation an initial GUI may be 
built but if the data structure or the data types 
are changed the manual adaption of the GUI is 


arduous. According to the changes of a data 
structure the extent of the redesign task may 
cause pretty much effort. 

• Due to the lack of adopted GUI design guide- 
lines, for similar data structures in different ap- 
plications a different GUI may exist which is con- 
tradictious to user interface consistency [10]. 

The approach introduced in this paper to address 
these problems is the automatic generation of GUIs 
from a high level specification. This generation is 
performed by a knowledge-based meta-tool which is 
used by a GUI designer. Questions which have to be 
tackled include the following: (1) Tb what extent can 
the designer be supported in the specification task? 
(2) What kind of user interface should the meta-tool 
have. (3) Which kind of knowledge is domain invari- 
ant and which is application specific (and therefore 
needs to be entered by the designer)? (4) Which set 
of default design decisions are adequate? 

Our approach to answer these questions is based 
on the following idea: The designer specifies data 
structures, data types and operations which the user 
of the target system has to perform with an user- 
friendly GUI. Corresponding GUI elements realizing 
these operations are associated automatically and the 
GUI is generated. The designer in turn refines the 
GUI by interactivly customizing the meta-tools asso- 
ciation and specifying qualitative layout constraints. 
This approach facilitates users who have no knowl- 
edge about interface programming to construct a GUI 
easily. Since the GUI of a meta-tool itself is in the 
domain our approch is applicable for the design of 
meta-tool’s GUI as well. 

In section 2 the addressed domain is introduced 
in more detail. Section 3 discusses the problems of 
configuration and generation of the target systems. 
In section 4 our approach is described to solve these 
problems. Section 5 compares our approach to re- 
lated work and section 6 gives some concluding re- 
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marks and perspectives on future work. 

2 Domain 

The domain our meta-tool addresses is the class of 
GUIs that allow the access of specified data struc- 
tures whose elements are characterized be specific 
data types. The access comprises additon, deletion, 
modification, selection and browsing of data struc- 
tures and instances. 

There exist rather different interpretations of what 
the notion GUI should mean [6]. In our meta-tool the 
GUI is built with a set of objects which have a de- 
scription of a graphical presentation and methods to 
handle the display presentation and the communica- 
tion with the underlying window system. Examples 
are buttons, settings or text fields. No other func- 
tionality is added to the GUI. The GUI objects are 
described within an object-oriented class hierarchy 
adopting inheritance. This is the common approach 
how state-of-the-art GUI toolkits and UIMSs are re- 
alized [6]. 

Our meta-tool produces specializations of classes 
in a class hierarchy provided by the GUI toolkit 
LispVtew [1] and instantiation methods. Lisp View 
provides an interfaces between Sun CommonLisp and 
OpenWindows. The same structure is generated by 
the GUI development system OpenWindows Devel- 
oper's Guide [3]. 

3 Problem Description 

The design of a meta-tool for automating the design 
of GUIs from specifications of data structures, data 
types and operations raises some questions which 
mainly influence the meta-tool design decisions: 

• Which kind of knowledge has to be represented 
to support the generation and which kind of 
knowledge representation should be used? 

• Which part of the knowledge is domain specific 
but application invariant and which part is ap- 
plication specific? 

• What kind of default configuration decisions 
makes sense? Can specific subdomains be identi- 
fied for which specific configuration macros may 
be used? 

• What is the most efficient way to enter geometric 
layout specifications? 


• Since an initially generated GUI in most cases 
does not meet the end user’s whishes rapid proto- 
typing facilities for iterative refinement and cus- 
tomization is needed. 

♦ The specification facility must allow only con- 
sistent specifications, Le. the designer’s specifi- 
cation has to be syntactically and semantically 
correct and the generator will produce a GUI 
inside the domain. How can we support specifi- 
cation consistency? 

The following section discusses our approach to- 
wards an automation of the GUI design addressing 
the questions given above. 

4 Approach 

State-of-the-art UIMSs mainly deal with a user- 
friendly composition of the GUI. From this point of 
view only the syntactical aspects in building GUIs 
are addressed. But naturally GUIs are built for user 
interactions which have certain semantics. Fbr in- 
stance, when the GUI designer using a direct ma- 
nipulation UIMS selects a button and arranges it 
in the target interface via mouse dragging he knows 
the reason why he selects a button and which opera- 
tion should be performed by clicking on the button. 
The GUI components are nothing else them graphi- 
cal presentations of abstract interactions. The map- 
ping from the semantics cf these interactions to cor- 
responding GUI elements is the main task of an GUI 
designer. 

Our approach for specifying GUIs starts from a se- 
mantic point of view and focuses on this mapping. 
The GUI designer does not specify a composition of 
the GUI components itself rather than the interac- 
tions the GUI components shall be used fbr. That 
means the focus of the specification is not how to 
present interactions on the screen but what kind of 
interactions shall be established. The interactions we 
consider are the access operations specified in section 
2. The mapping from the interaction specification 
to the GUI components is done by the meta-tool 
automatically. In a further step the designer may 
customize the generated GUI either by changing the 
mapping or specifying additional qualitative layout 
constraints. 

4.1 Configuration Process 

In this section the configuration process is discussed. 
Figure 1 provides an overview of the configuration 
steps. The actions the designer has to perform are 
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Figure 1: iterative GUI configuration process 

specification and evaluation represented as round- 
cornered boxes. The meta-tool activity (the genera- 
tion of the GUI) is represented as a rectangular box. 

The designer starts with the specification of the 
desired interactions on data structures. Then an ini- 
tial GUI is generated by the meta-tool using default 
mapping and layout configurations stored in a knowl- 
edge base (see section 42). The initial generation has 
to be evaluated by the designer. Then cne of the fol- 
lowing four choices may be made: 

1. The designer agrees with the generated GUI and 
the configuration process is finished. 

2. The designer specifies qualitative geometric lay- 
out constraints to rearrange the GUI compo- 
nents on the screen. 

3. The designer alters the mapping between the 
specified interactions and the corresponding GUI 
component. 

4 The designer manipulates the interaction speci- 
fication, e.g. a new element is added to a data 
structure. 

In case cf a new or re-specification a new genera- 
tion cycle starts. The order given above implies the 
extent of the GUI redesign in a cycle after evalua- 
tion. Choice 2 affects only the geometric position of 
GUI elements, choice 3 affects the presentation of an 
interaction, and choice 4 affects the interaction it- 
self. Explorative rapid prototyping by iterating the 


configuration cycles is supported conveniently, since 
the designer starts with a specification of abstract in- 
teractions omitting GUI aspects in the initial phase. 
In following cycles he can customize presentation as- 
pects very quickly or redesign the interactions. 

Since end users are supposed to design the GUIs 
the meta-tool must provide user-friendly graphical 
interfaces itself. Tb support specification consistency, 
the specification is menu-driven as far as possible. 
Menus with appropriate selections may be offered 
which is further discussed in section 4.2. An interest- 
ing issue is that the GUI of the meta-tool to enter the 
specification is itself in the domain of the meta-tool. 
Since the meta-tool allows to use the specification 
languages directly without the corresponding GUI 1 , 
the GUI for the meta-tool can be generated by the 
met a- tool itself. 

4.2 Configuration Knowledge and 
Representation 

This section deals with the knowledge needed to au- 
tomate the GUI configuration. We distinguish two 
classes of knowledge. Knowledge is needed to support 
an efficient user-friendly specification and to gener- 
ate a GUI with an minimal specification. This kind 
of knowledge is application-invariant and refered as 
domain -specific (in the GUI domain). On the other 
hand application -specific knowledge must be enter d 
by the designer to build an GUI for a set cf certain 
interactions. The following two subsection discuss 
these two knowledge classes. 

4.2.1 Domain-specific knowledge 

The following Ested knowledge categories are stored 
in the meta-tool’s knowledge base in order to sup- 
port specification and generation. Note that this 
b mainly knowledge about the possible application- 
specific knowledge (e.g. possiple types of layout con- 
straints) and therefore meta-knowledge. 

• model of target architecture; the structure 
of the code generated by our meta-tool is given 
by the code structure the Developer’s Guide for 
LispView interfaces [3] generates. 

• a library of interaction types and data 
types; interaction types include read and write 
access to data and selection of data. Cur- 
rently the Ebrary of data types includes enu- 
meration, character, real, integer, string, sym- 
bol, and object-class. 

1 Otherwise there would be mete-tool tower never aiding. 


52 





Gastner 


• a library of GUI elements; this library is 
given by the used GUI toolkit Lisp View [1], 

• mapping of interaction specifications to 
GUI elements; the mapping is stored as a ma- 
trix in which for certain conditions made in a 
data type specification a set of possible GUI 
components is associated. The GUI component 
selected by default is marked (see also section 
4.4. 

• library of layout constraints; currently 
we have realized 36 layout constraint types 
which are hierarchically organized and offered in 
menus. Furthermore, there exits a layout con- 
straint construction facility for the meta-tool de- 
signer to implement additional constraint types 
based on a combination of types from a basic set. 

• standard configurations; see section 4.4. 

The domain-specific knowledge is stored in ASCII- 
files in special representation languages. The files 
may either be edited directly by a text editor or be 
generated from graphical specifications. An inter- 
preter reads these files and maps the external rep- 
resentation to internal objects. 

4.2*2 Application-specific knowledge 

As shown in figure 1 there are three specification pos- 
sibilities providing input for the generator. 

• interactions; the specification comprises the 
type of operation and the data type to be ac- 
cessed. The data type is specified separately. 
Thus more than one interaction may access data 
of the same data type in different ways. The ex- 
ample below shows the declarative specification 
generated from the graphical specification envi- 
ronment. A manipulation interaction is specified 
on data of an integer slot. The value range is re- 
stricted between 100 and 500, the slot is angle- 
valued, and the value must be unique and en- 
tered. 


(dsf- interact ion 

:id ’sngixis-numbsr-manipulstion 
: ops r it ion Manipulation 
:diti-typ* 'ongino-numbor-typo) 
(dif-intigir 

:id Mngino-numbor-typo 
:oqualorgr«ator 100 
slassoroqual GOO 
in card 1 
»axcard 1 
:uniqu« T) 


The declarative specification languages may also 
be used directly by the designer. Both inter- 
actions and data types are offered in menus to 
the designer. The menus are configured dy- 
namically according to certain specification con- 
straints; e.g. the following constraint may not be 
violated in the example above: (lessorequal 

min card maxcard). 

• association of interactions and GUI com- 
ponents; if the designer does not agree with the 
meta-tool’s association he may select another 
association or more than one associations for a 
given interaction from a menu. The menu items 
consist of all GUI components which are accept- 
able presentations for the interaction asserting 
a consistent specification. If the designer asso- 
ciates more than one GUI component to an in- 
teraction, the interaction is presented in different 
fashions in the GUI. For instance, the interac- 
tion in the example above may be presented as 
numeric field or a slider. The meta-tool would 
select the numeric field by default. 

• layout constraints; the qualitative layout con- 
straints may be specified using a declarative 
specification language <x a GUI generating sen- 
tences of this language. The following example 
demonstrates the power and user-friendlyness of 
our layout mechanism: 

Let B\ , Bj , . . B b be boxes which shidl be arranged as 
follows: B a and B*> shall be at the bottom of the layout 
frame; B\ shall be m the upper kft corner cf the layout 
frame; and B3 shall be over B3 and Bj ■ This is expressed 
as follows: 

(bottom-margin £< B& ) 

(upper- left -comer B\ ) 

(over B3 (Ba B* )) 

Entering the first layout constraint via the specification 
GUI B4 and then B& would by selected with the mouse cn 
the screen and then the constraint bottom -margin would 
be selected from the menu. 


4.3 Layout computation 

Each GUI dement has a rectangular bounding-box 
which provides the size for the layout generator. The 
36 layout constraints are one-dimensional geometric 
relationships between these boxes. N-ary relation- 
ships are resolved into tanary ones which are con- 
nected with a conjunction. The corners cf the boxes 
are represented by variables and the constraints are 
always inequations of the following form: 

Cti < Xj - x, < A 

These unequations can be solved using a kmgest-path 
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algorithm suggested in [11]. If there are inconsisten- 
cies in the specified constraint set our algorithm re- 
tracts contradictious constraints. The boxes are ar- 
ranged fulfilling the specified constraints and are po- 
sitioned in the upper left corner of the layout frame. 
In a second cycle overlapping boxes (this may oc- 
cur if the constraint set is not restrictive enough) 
are solve by adding additional contraints with dis- 
junctions: A box B\ and a box B 2 do not over- 

lap if (beside B 1 B 2 ) V (beside B 2 B 1 ) V (over 
B\B 2 ) V (over B 2 Bi) holds. Since there may be 
a huge number of layout configurations solving the 
constraint set without overlapping the layout algo- 
rithm gets a certain time for processing (e.g. three 
seconds). The algorithm generates a set of solutions 
and then selects the best solution when the time is 
over. The selection criteria adopted currently is ei- 
ther to minimize the area of the layout frame if the 
size is not prespecified or to arrange the boxes with 
equal distances between them in a fixed layout frame. 

4.4 Standard Configurations 

In order to give support in the specification of GUI 
component associations to interactions and to select 
default associations we (partly) represent knowledge 
found in the OPEN LOOK application style guide- 
lines [2]. This knowledge is stored in a matrix in this 
way that for each GUI component it is marked under 
which conditions it is appropriate and if it should be 
selected by default. Furthermore, OPEN LOOK pro- 
vides a unique look-and-feel for all the target GUIs 
and the GUI sepecification environment of our meta- 
tool. 

It is possible to preconfigure special editor types 
which include a number of fixed interactions. For 
instance, a login editor consists always of two inter- 
actions, one for entering the user’s name and one for 
entering the password. These two interactions are 
preconfigured as a symbol and string manipulation 
interaction. Furthermore, a layout frame with a fixed 
size is configured, layout constraints are specified that 
both GUI components (the meta-tool will associate 
two text fields) should be centered and the text field 
for the user’s name should be located over the field 
for the password entry. The configuration is stored 
as subclass of a preconfigured editor class. Other 
specialized editors may be partly preconfigured and 
layouted like object editors or browsers. Preconfig- 
ured GUI classes can be dynamically added by the 
designer. 

Adopting this configurartion library and the repre- 
sented OPEN LOOK style guidelines we facilitate the 


generation of GUIs which have a common structure 
and supports GUI consistency [10], 


4.5 Implementation 

Our meta-tool is implemented in Sun CommonLisp, 
CLOS and LispView [1]. Object-oriented program- 
ming is adopted basically. The target code is gener- 
ated using templates which are expanded according 
to the designer’s specification or standard configura- 
tions. By replacing the templates it is possible to 
generate other GUI target code as well. 


5 Related Work 

In the last decade human-computer interaction and 
the user interfaces have become an important re- 
search field. UIMSs try to improve GUI development 
and support mechanisms for GUI and dialogue spec- 
ification, representation and management [6] [9]. In 
[7] several generations of UIMSs are identified. It 
is predicted that future UIMSs will be knowledge- 
based and generate a user interface automatically us- 
ing the specification of the underlying application. 
Our approach is a step in this direction. Currently 
the interactions have still to be coded by a GUI de- 
signer, but there should be a way to generate the 
interaction specification from application programs 
automatically as well. 

A number of development methodologies have been 
suggested for user interfaces. Most of them claim 
explorative prototyping as our approach (see figure 
1), e.g. the star life cycle suggested in [7]. 

User interfaces may be specified language-based 
with special user interface description languages, 
graphical-based with direct manipulation facilities or 
with automatic generation from interaction descrip- 
tions [9]. Since our meta-tool generates code which 
can be manipulated by the Developer’s Guide [3] 
our approach combines these three possibilities which 
may be alternatively used. 

Similar approches for automatic generation of 
GUIs are used in the GADGETS system [8] and 
the PRED system [13], but they lack qualitative 
layout specifications. Automatic presentation sys- 
tems for information like SAGE [12] also use meta- 
information to select an adequate presentation style. 
A similar approach of default configurations of edi- 
tors is applied in the meta-tool DOTS [4]. 
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6 Concluding Remarks and 
Future Work 

We suggested an approach towards automation of 
user interface design which starts from a semantic 
point of view. The initial specification only deals 
with what the GUI is to be built for and not how . Fur- 
ther prototyping cycles allow to customize the gen- 
erated GUI qualitatively. Since the generated GUI 
code is interpretable by the direct manipulation tool 
Developer’s Guide [3], also quantitative layouting is 
available and may be adopted alternately. Since the 
meta-tool’s GUI is in the meta-tool’s domain itself a 
reflexive application of the meta-tool is possible. 

In the project KME (Knowledge Maintenance 
Environment) 2 we designed a meta-tool called KME 
workbench [5] for generating maintenance compo- 
nents for knowledge bases of expert systems. A main- 
tenance component for updating objects of an object 
oriented representation needs a GUI of the domain 
described in this paper. Thus the GUI design meta- 
tool is part of the KME workbench. We experienced 
in this project that qualitative layout specifications 
are very convenient and allow rapid explorative pro- 
totyping. The GUI specification environment also al- 
lows end users (e.g. knowledge engineers with only 
few programming experience) to build adequate GUIs 
easily. 

We acquired GUI design knowledge from the 
OPEN LOOK GUI application style guidelines [2] 
which is represented in a matrix representation and 
allows the meta-tool to provide default configura- 
tions. Furthermore, the explicit representation can 
easily be changed and augmented. 

Currently we work on the extension of default con- 
figurations and GUI facilities. Special editor types 
are identified in more specific application domains 
and represented. We will evaluate how the GUI spec- 
ification rap be acquired automatically from the un- 
derlying application. In the knowledge maintenance 
context we will try to generated a default dialogue 
control supported by a transaction management. 
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